1. Field of the Invention
The invention relates to a picture coding apparatus and a decoding apparatus which, when a video signal is to be encoded and decoded for storage or transmission, can realize high efficiency coding, and also a high speed search or reproduction of excellent efficiency.
2. Related Art of the Invention
Recently, MPEG (Moving Picture Expert Group) which is an international standard is typical of video coding.
According to MPEG, intrapicture coding is first conducted, motion vectors of a P-frame or a B-frame are detected on the basis of the intrapicture-decoded picture (I-frame) corresponding to the coding, motion compensation prediction of the detected motion vectors is conducted based on the motion vectors, and a difference between a motion compensated picture obtained by the motion compensation and the picture to be encoded is encoded. Namely, MPEG is a coding method of the intrapicture/interpicture selection type in which the intrapicture-decoded picture and the intrapicture coding of the picture are combined with each other.
In a coding method of the intrapicture/interpicture selection type, in order to prevent error propagation from occurring or enable an encoded picture to be reproduced from the middle irrespective of the contents of a moving picture, intrapicture coding is conducted with a predetermined period. When a moving picture is to be subjected to high speed reproduction, therefore, only a picture which has undergone intrapicture coding is extracted and then decoded to obtain a reproduced picture.
FIG. 10 is a block diagram of a high-speed reproduction apparatus in which an intrapicture-coded picture is extracted and then decoded. In the figure, 1 designates a coded input signal, 2 designates intrapicture-coded picture extracting means, 3 designates an intrapicture-coded signal, 4 designates intrapicture-coded picture decoding means, and 5 designates a decoded output signal.
The operation of the thus configured high-speed reproduction apparatus will be described. The intrapicture-coded picture extracting means 2 extracts only a picture which has undergone intrapicture coding, from the coded input signal 1 which has been subjected to coding of the intrapicture/interpicture selection type, and outputs the intrapicture-coded signal 3. The intrapicture-coded picture decoding means 4 decodes the intrapicture-coded signal 3 and outputs the decoded output signal 5. In this way, high-speed reproduction is conducted by extracting only an intrapicture-coded picture and decoding it.
In the high-speed reproduction apparatus, however, a P-frame or B-frame cannot be used in high-speed reproduction unless a I-frame or P-frame which is to be referred is decoded. Consequently, it is difficult to produce a high-speed reproduction apparatus which can smoothly conduct high-speed reproduction.
Even when only a part of a picture is updated, moreover, the previous picture must be updated. This causes the amount of information to be processed for high-speed reproduction, to be increased.
Since reproduction is performed at fixed intervals, it is difficult to conduct high-speed reproduction or a high-speed scene search in accordance with a change in contents of a picture.
Coding of pixel value information for a conventional picture scene is conducted on a single layer. In other words, pixel value information of the whole contents of one scene of a picture are completed on a single layer, and coding is conducted on each of the pixel value information on the single layer. When pixel value information which is completed on a single layer is to be encoded, the need for efficiently conducting the coding is growingly increased.
However, conventional means for encoding pixel value information which is completed on a single layer has a problem in that, even when a pixel which is not required to be encoded exists in the single layer, coding must be conducted with considering the value of the pixel and therefore the amount of information to be encoded is increased so that the processing speed is reduced.
According to the invention, a technique of layered coding is employed in which a moving picture is separated into layers such as a background, a foreground, and characters or pictures are separately prepared as one layer and coding is conducted on each layer. As a novel technique of coding a moving picture, the inventors have proposed in another patent application a technique of layered coding which uses a representative picture structure and global motion compensation.
FIG. 11(a) is a diagram illustrating a layered structure. A final output picture is generated by synthesizing a foreground layer with a background layer. In each layer, coding is conducted by using a representative picture (template) structure and global motion compensation. Global motion compensation is a method in which motion of the whole picture is represented by global motion parameters such as translation, rotation, zooming, and reduction, motion compensation is conducted, and a predictive coded picture is generated. In the method, a predictive differential signal may be encoded or predictive differential signal coding may be omitted.
FIG. 11(b) is a diagram illustrating the representative picture structure. Each layer is tracked along the time series, and a picture which represents each scene is selected as a representative picture. Examples of selecting a representative picture are listed below:
1. A picture which, in motion compensation, exhibits a minimum differential signal after motion compensation is selected.
2. A picture which is easy to search in a search process is artificially selected.
3. The first picture in a scene is selected.
In each scene, global motion compensation is conducted while using the representative picture of the scene as the reference picture, and a predictive coded picture is generated. As the representative picture to be referred, a representative picture of the scene, or a picture which has been selected as the representative picture in a previous scene may be selected. In other words, there is a possibility that the same representative picture is referred in a plurality of scenes. Such a scene may be a scene between scene changes in a picture, or a small scene which is obtained by partitioning in accordance with the groups of motions of the object. Alternatively, the scene may be deemed as a scene which is united in contents of a picture.
To comply with this, it is an object of the invention to provide a picture coding apparatus and a decoding apparatus for the coding apparatus which can conduct a search or reproduction wherein stress is placed on rapidness or wherein stress is placed on rapidness and also on continuity.
It is another object of the invention to provide a picture coding apparatus and a decoding apparatus for the coding apparatus which can efficiently encode and decode a picture signal.
Specifically, an object of the invention is to provide a picture decoding apparatus which can conduct high speed reproduction rapidly and efficiently with using a representative picture structure and a layered structure in layered coding which uses a representative picture structure and global motion compensation.
In the picture coding apparatus of the invention, even when the same representative picture is referred plural times in a plurality of scenes in the case where a coded signal consisting of representative pictures representing scenes of a picture and predictive coded pictures referring the representative pictures is to be encoded, reference relationships are encoded as additional information, and therefore it is possible to realize high-efficiency coding in which waste such as duplication of encoding the same representative picture is eliminated. In addition to the reference relationships, significance in contents of the time series of representative pictures may be encoded so that, in decoding associated with a high speed search or high speed reproduction, only a representative picture of higher significance is allowed to be decoded only by referring additional information of higher significance.
In the picture decoding apparatus of the invention, when a coded signal consisting of representative pictures representing scenes of a picture and predictive coded pictures referring the representative pictures is to be decoded, only a coded representative picture signal is extracted from a coded input signal by representative picture extracting means and then decoded by representative picture decoding means. Since only a representative picture signal is subjected to decoding, high-speed reproduction can be realized. Since a representative picture signal is selected in accordance with a large change of a picture, the selected signal corresponds to a scene change of a picture. Therefore, high speed reproduction or scene search of excellent efficiency which reflects the picture contents is enabled only by tracking representative pictures.
In the picture decoding apparatus of the invention, when a coded signal consisting of representative pictures representing scenes of a picture and predictive coded pictures referring the representative pictures is to be decoded, significance of a representative picture in a time series of a picture is set in decreasing order of significance in contents, representative picture extracting means extracts a representative picture in consideration of significance, and only a representative picture of higher significance is decoded. Since significance is considered, it is possible to omit decoding of a representative picture which is not important in contents. Therefore, high speed reproduction or search of more excellent efficiency is enabled.
In the picture decoding apparatus of the invention, when a coded signal consisting of representative pictures representing scenes of a picture and predictive coded pictures referring the representative pictures is to be decoded, high speed reproduction is performed by conducting not only decoding of a representative picture but also a part of a predictive coded signal referring a representative picture signal. It is sufficient for decoding of a predictive coded signal to decode a representative picture signal to be referred and global motion parameters. Since high speed reproduction is performed while decoding a part of a predictive coded signal, it is possible to realize high speed reproduction which complies with the picture contents and in which motion is relatively smooth.
In the picture decoding apparatus of the invention, when a coded signal consisting of representative pictures representing scenes of a picture and predictive coded picture referring the representative pictures is to be decoded, the time interval between adjacent representative picture signals, namely, predictive coded pictures existing between representative picture signals is counted. The coded signal is decoded while shortening the representative picture interval to a time interval which is proportional to the number of the predictive coded pictures, and then output, thereby realizing high speed reproduction which maintains the original smoothness in the time direction of the picture.
In the picture coding apparatus of the invention, when a picture is to be encoded for each of layers such as a background and a foreground, significance in contents is added to each layer as additional information and encoded. In decoding for high speed reproduction or scene search, therefore, the significance is referred so that a layer of higher significance is subjected to decoding which is detailed in contents, and that of lower significance is subjected to simplified decoding or refrained from being subjected to decoding.
In the picture decoding apparatus of the invention, in the case where a picture is encoded for each of layers such as a background and a foreground, when decoding for high speed reproduction or scene search is to be conducted, additional information of layer significance of a coded signal of each layer which is an input signal is referred to judge significance. Then a layer of higher significance in contents is subjected to decoding which is detailed, and that of lower significance is subjected to simplified decoding or refrained from being subjected to decoding, thereby realizing decoding in which waste is eliminated and which enables a high speed scene search or reproduction of excellent efficiency.
In the picture decoding apparatus of the invention, in the case where a picture is expressed by a layered structure and a coded signal consisting of representative pictures representing scenes of a picture and predictive coded pictures referring the representative pictures is to be decoded, when decoding of a representative picture signal is newly conducted in layers the number of which is larger than a predetermined one by predetermined-number judging means, the output of a representative picture is updated in all layers. Even when coding positions of representative pictures in time series in layers are different from each other, therefore, it is possible to coincide output positions with each other. When the number of layers is less than the predetermined number, the output of a representative picture is not updated. Consequently, processing can be simplified.
In the picture decoding apparatus of the invention, in the case where a picture is expressed by a layered structure, the method of decoding layer-1 is changed from that of decoding layer-2, and the output process is performed without synthesizing layers with each other, but with switching the output layer, thereby realizing high speed reproduction or scene search of excellent efficiency in which functions of high speed reproduction and high speed scene search are selected in accordance with the picture contents of the layers. Since decoding is required to be conducted only in the switched layer, decoding in which decoding of a non-selected layer is omitted or waste is eliminated is enabled.
Specifically, an object of the invention is to efficiently realize coding and decoding in a picture which is expressed by a layered structure, with using transparency information between layers of the layered structure.
In the picture coding apparatus of the invention, coding of transparency of a pixel value obtained from a picture signal results in most cases in a distribution in which the ratio of intermediate level values is small or most elements are 0% or 100%. When the binarization of transparency is efficiently conducted so that coding of intermediate level values is omitted or simplified, therefore, the coding efficiency can be improved.
In the picture coding apparatus of the invention, when the frequency ratio of intermediate level values is not small, the part other than that of transparency of 100% is extracted as one region, and pixels in the one region are subjected to a multivaluing process, whereby the process of multivalue-encoding the region other than the one region can be omitted. Consequently, the coding efficiency can be improved.
Specifically, the invention is a picture coding apparatus which judges for a picture signal on the basis of the transparent state of the pixel value whether a pixel region is to be encoded or not, and a pixel region to be encoded is encoded by a predetermined coding method on the basis a pixel to be encoded position signal which is obtained by encoding the judgment result.
In the picture coding apparatus of the invention, a picture signal is checked to judge whether a pixel region is to be encoded or not, on the basis of the transparent state of the pixel value, and a pixel region is to be encoded is encoded by a predetermined coding method on the basis a pixel to be encoded position signal which is obtained by encoding the judgment result. Therefore, coding of a pixel region other than the pixel region to be encoded can be omitted. Consequently, the invention has an advantage that the efficiency of coding can be improved.
To achieve the above objects, the invention provides a picture coding apparatus comprising: binarizing means (2002) for binarizing transparency information for each of pixels of a picture on the basis of a predetermined value; binary coding means (2004) for binary-encoding the transparency information binarized by the binarizing means; judging means (2203) for judging whether or not the transparency information for each of the pixels is to be multivalue-encoded on the basis of the transparency information, for each of the pixels; and multivalue coding means (2206) for multivalue-encoding the transparency information which is judged by the judging means that the transparency information is to be multivalue-encoded.
Further, to achieve the above objects, the invention provides a picture coding apparatus comprising: judging means (3005) for judging whether or not each pixel of a picture is to be encoded on the basis of transparency information for the pixel; and coding means for encoding each pixel which is judged by the judging means that the pixel is to be encoded.
To achieve the above objects, the invention provides a picture coding apparatus comprising: representative picture selecting means (12) for selecting a representative picture from a picture group which includes a plurality of pictures; coding means for encoding each of the pictures which are included in the picture group, with using the representative picture; and signal generating means (17) for generating a signal including the pictures encoded by the coding means, and for generating a signal including coding information relating to the representative picture which is used for each of the encoded pictures included in the signal; the signal generating means being able to separately send the signal including the encoded pictures and the signal including the coding information relating to the representative picture.
Further, to achieve the above objects, the invention provides a picture decoding apparatus comprising: representative picture extracting means (102) for extracting information relating to all or part of the representative pictures, from the signal generated by the signal generating means (17) of the picture coding apparatus; and representative picture decoding means (104) for decoding the representative picture extracted by the representative picture extracting means.